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ABSTRACT 



A study to determine to what extent course evaluation 



reports provided by students serving on course liaison committees 
accurately reflect the perceptions of the entire student body is 
presented. A system of liaison committees composed of student 
volunteers was developed at the University of Iowa Medical School to 
meet regularly with course directors and provide ongoing feedback 
during the course and a summary report to the course director and 
curriculum committee at the conclusion of the course. The courses 
taken by first year medical students in the fall semester were 
selected for study. A questionnaire divided into three parts 
accompanied the liaison report. Students were asked to evaluate their 
general perceptions of the accuracy of student liaison reports, 
evaluate the accuracy of the report itself after reviewing it, and 
provide general recommendations regarding the use of student liaison 
reports for the purpose of curriculm:: development. According to the 
results, reports produced by the liaison committee: (1) are moderate 
to very accurate in reflecting the ratings of the majority of 
students; (2) would be acceptable to the majority of students as 
adequate representation for making suggestions for course and 
curriculum changes; and (3) will tend to have a somewhat positive 
bias for courses in which there may be intense negative sentiment. 
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^ To determine to what degree course evaluation reports provided by volunteer 

Q students serving on course liaison committees accurately reflect the perceptions of the 



larger student body. 
Perspectives 

Obtaining useful feedback regarding how a course is functioning at a time when 
corrections in the course can still be made is no small problem under the best of 
circumstances; and in the case of multi-disciplinary and mulii-instnictor courses is 
substantially more complicated. This probably accounts for why such formative evaluation 
is not often performed. Instead, post-course computer-processed summative evaluations 
are the most common form of course evaluation (Braskamp, Brandenburg, Ory, 1984). 
Among the limitations of such end of course evaluations are that they are obtained at a 
time when their only influence can be on future course offerings. Also, because of the 
computerization, student responses must be from a prescribed number of options. 
Although providing room for comments h recommended and commonly done, students 
usually limit their responses to the selection of one of the options. The sum effect is that, 
for most courses, once they are in motion, constructive changes are unlikely to occur until 
the next course offering. 

In order to gather more timely and more detailed feedback from students, a system 
of liaison committees composed of student volunteers was developed at our medical school. 
The liaison committees meet regularly with course directors to provide ongoing feedback 

U 

during the course and at the conclusion of the course provide a summary report to the 
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The use of student volunteers for such committees has proven to be necessary 
because of the extensive commitment students must make to the effort. Using random 
assignment of students to committees is not likely to produce committees with the degree 
of commitment to their task that will be obtained with student volunteers. Because the 
liaison committees are composed of volunteers and the feedback and reports they provide 
are heavily relied upon, the extent to which these reports reflect the opinion of the entire 
student body is a major issue. The purpose of this study was to determine to what extent 
student liaison reports accurately reflect the perception of the entire student body. 
Method 

The courses taken by first year medical students in the fall semester were selected 
for study. The courses evaluated were: anatomy, biochemistry, biostatistics, embryology, 
and histology. (To enable a frank discussion of the results yet maintain confidentiality, the 
courses will, hereafter, be referred to only by a randomly assigned number.) It was 
considered essential to the validity of the assessment of the accuracy of the reports that the 
students who are doing the evaluation should be able to review an actual report before 
making their judgments. Since these reports were fairly extensive, sometimes four or five 
pages of single spaced type, it would be too much of a burden to have each student 
evaluate z\\ courses. It was also considered important that results from more than one 
course should be represented as the students comprising liaison committees varied by 
course. For these reasons, a multiple matrix sampling approach was adopted. This 
involved dividing the entire first year medical class (total N = 178) into five random groups 
(one group for each of the five Cfnirses taken during the semester). Each randomly 
determined group was assigned a different liaison committee report to evaluate. 

The questionnaire accompanying the liaison report was divided into three parts. 
The first part asked students to evaluate their general perceptions of the accuracy of 
student liaison reports. This was to be completed before reviewing the actual report and 
was included partly as a check on the equivalence of the randomly determined groups. The 
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second part of the questionnaire asked students to evaluate the accuracy of the report itself 
after reviewing the report. The fii.al section asked general recommendations regarding the 
use of student liaison reports for the purpose of curriculum management. Smdents 
responded primarily |on a five-point rating scale or yes/ no depending on the nature of the 
question asked. Thelsurveys were distributed within six weeks of the end of the courses 
being evaluated. There were three follow up mailings to non-respondents. 

In order to test differences between the groups in their response to the 
questionnaire, two types of analyses were performed. For questions vnth five options 
representing gradations of response, a one-factor analysis of variance was computed with 
means and standard deviations (SD) reported as descriptive statistics. Since a five-option 
question is likely to have scale properties that make it a questionable application of 
analysis of variance procedures, chi square analyses were also computed for such items. In 
cases where the two analyses did not lead to the same conclusion, both are reported. This 
served to ensure that any differences noted were not simply due to a violation of the scaling 
assumptions of analysis of variance while enabling the use of means and standard 
deviations as descriptive statistics instead of cumbersome distributional statistics. For 
yes/no questions only chi square procedures were employed with percentage of the sample 
selecting yes serving as the primary descriptive statistic. 
Results 

Response rates ranged from 72% to 80% for the five different randomly determined 
groups (N=26-28 out of 35 or 36 in each group). These were considered good response 
rates considering the magnitude of the task required of respondents. 

The general perceptions of the accuracy of student liaison reports and student 
comfort with their use for course and curriculum decisions are shown in Table 1. These 
perceptions were fairly neutral with means ranging from 3.23 to 3.77 for the five groups and 
averaging 3.51 (1 = Not at All, 5 = Extremely Accurate). Student level of comfort with 
using the liaison committee reports as the primary source of student input regarding course 
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decisions was relatively high with from 50% to 75% indicating that their interests would be 
served. For making curriculum decisions, however, students were less comfortable with the 
liaison reports. The percentages indicating their interests would be served declined to 
between 38% to 61%. Differences between the five randomly determined sub-groups were 
not statistically different, suggesting that the groups were relatively similar in their pre- 
existing perception regarding the liaison committees. 

Table 2 shows the results from part 2 of the questionnaire in which students rated 
the liaison committee report which accompanied the questionnaire. Mean scores ranged 
from 3.2 to 4.1 suggesting a moderate to very positive evaluation of the accuracy of the 
reports. The differences in ratings of reports from the different courses were, however, 
statistically significant (p < .01). This indicates that not all of the liaison committee reports 
were equally accurate. When asked to indicate how their own assessment of the course 
would compare with that of the liaison committee report, four out of the five groups 
indicated that on average .heir evaluations would be more negative. The differences 
among the groups in these ratings were statistically significant (p < .013). An examination 
of these results in terms of the number of students selecting each response found that even 
though the means tended toward one end of the scale or the other, a majority of the 
respondents indicated their own evaluation of the course would have been the same as the 
liaison committee report for four of the five courses. In the lone exception (course #2), 
59% of the respondents indicated that their own evaluation would be more or much more 
negative. This particular course had a long history of student dissatisfaction and the report 
itself was one of the more negative. 

In spite of this perception, 78% of the students evaluating course #2 report 
indicated that their views would be adequately represented if the liaison report was used as 
the primary source of student input regarding the course. For all courses, the percentages 
exceeded 50% (range 57 to 88%) indicating that overall, the liaison reports adequately 
represent the majority viewpoint of the entire student body. This suggests that liaison 
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committee reports reflect student opinions in direction and substance, but in cases of 
intense smdent dissatisfaction may tend to under-report the intensity of the dissatisfaction. 

Results from the final section of the questionnaire (Table 3) asked students to 
indicate their general perceptions of the student liaison reports in view of having actually 
read a report. Regarding general curriculum decisions, the percentage who responded that 
the'r interests would be served if the liaison reports were relied upon increased from 50% 
before reviewing the liaison report to 70% after reviewing tlie report. In terms of accuracy 
ratings of the reports, the before and after means were almost identical. Thus actually 
reviewing a student liaison committee report had the effect of boosting student confidence 
in the validity of these reviews, although perceptions of the accuracy of the reports were 
not greatly influenced. 
Educational Importance 

Student liaison committees composed of volunteers offer a convenient and readily 
accessible source of student input regarding ongoing or planned course activities. This 
study was conducted to determine to what degree information obtained from these 
volunteer groups accurately reflects the opinion of the larger student body. Based on this 
study, it can be conclufJed that the reports produced by the liaison committee: 1) are 
moderate to very accurate in reflecting the ratings of the majority of students, 2) would be 
acceptable to the majority of students as adequate representation for making suggestions 
for course and curriculum changes, and 3) will tend to have a somewhat positive bias for 
courses in which there may be intense negative sentiment. 
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TABLE 1 

QUESTIONNAIRE PART I RESULTS: GENERAl. PERCEPTIONS REGARDING 

LIAISON COMMITTEE REPORTS 

I. 1. In generj»!, how accurately do you think the liaison committee reports reflect 
the perceptio'ts of your entire class? 



(1 = Not at all, 2 = somewhat, 3 = moderately, 4 = very, 5 = extremely) 



Course # 


N 


Mean 


SD 


1 


26 


3.23 


.76 


2 


25 


3.32 


1.03 


3 


24 


3.75 


.94 


4 


26 


3.77 


.76 


5 


24 


3.50 


.78 


TOTAL 


125 


3.51 


.88 



^(4 120) P (There were 10 no opinion responses omitted from this analysis) 

2. Percentage of students who responded that their interests would be served if 
liaison committees were relied upon as the primary source of student input 
regarding: 



a) Individual Courses b) Overall Curriculum 



Course # N % Yes % Yes 

1 28 68% 43% 

2 27 70% 38% 

3 26 65% 50% 

4 28 75 % 61% 

5 26 50% 58% 
TOTAL 135 65.9% 50% 

X2(4) = 4.25, p< .38 X2(4) = 3.86, p< .426 
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TABLE 2 

Questionnaire Part II Results: Assessment of Actual Liaison Committee Reports 

II. 1. Degree of agreement with the comments and recon.mendations contained in 
the report. (1 = Not at all, 2 = Somewhat, 3 = Moder^t^'.y, 4 = Very, 5 = 



Extremely) 

Course # N Mean SD 

1 28 3.21 .92 

2 27 3.63 1.01 

3 25 4.12 .88 

4 28 3.89 .74 

5 26 3.65 .89 

TOTAL 134 169 093 



P(4,129) = 3.86, p< .0054 



2. How would your own evaluation of the course compare with that contained 
in the attached report? (1 = Much more positive, 2 = More Positive, 3 = Same, 4 
= More Negative, 5 = Much More Negative) 



Course # N Mean SD 



1 28 3.32 .67 

2 27 3.67 .62 

3 24 3.04 .46 

4 28 2.89 .63 

5 26 3.08 .74 



TOTAL 133 3.20 .68 



F(4,128) = 6-16, p< .0001 
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TABLE 3 



QUESTIONNAIRE PART III: POST REVIEW GENERAL PERCEPTIONS 

III. 1. Regarding general curriculum decisions such as course sequencing and 

student evaluation methods, would your views be adequately represented if liaison 
committee reports were relied upon as the primary source of student input? 



Course # % Yes 

1 53.6 

2 77.8 

3 76.9 

4 66.7 

5 76.9 
TOTAL 701 

X2(4) = 5.72,p< .221 

2. After reviewing the attached liaison committee report, hov "curately do you 
think the reports reflect the perceptions of your entire class? 



Course # N Mean SD 

1 24 3.08 .72 

2 27 3.33 .78 

3 24 3.92 .7? 

4 28 3.68 .72 

5 25 3.48 .82 
TOTAL 128 3.50 .80 

F(4,123)= 4.26, p< .0029 
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